AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.687821	1.67
	2_DecisionTree	Decision Tree	logloss	0.436436	16.45
the best	3_Default_Xgboost	Xgboost	logloss	0.259062	26.31
	4_Default_NeuralNetwork	Neural Network	logloss	0.362407	10.07
	5_Default_RandomForest	Random Forest	logloss	0.380921	24.85
	Ensemble	Ensemble	logloss	0.259062	1.62

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: binary:logistic
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

24.7 seconds

Metric details

	score	threshold
logloss	0.259062	nan
auc	0.958834	nan
f1	0.904165	0.452821
accuracy	0.893487	0.556852
precision	0.996491	0.994703
recall	1	0.000147622
mcc	0.785646	0.556852

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.259062	nan
auc	0.958834	nan
f1	0.902181	0.556852
accuracy	0.893487	0.556852
precision	0.914141	0.556852
recall	0.890529	0.556852
mcc	0.785646	0.556852

Confusion matrix (at threshold=0.556852)

	Predicted as long	Predicted as short
Labeled as long	1779	204
Labeled as short	267	2172

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

14.9 seconds

Metric details

	score	threshold
logloss	0.436436	nan
auc	0.870434	nan
f1	0.825652	0.518668
accuracy	0.79896	0.518668
precision	0.791353	0.518668
recall	1	0.0630936
mcc	0.592415	0.518668

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.436436	nan
auc	0.870434	nan
f1	0.825652	0.518668
accuracy	0.79896	0.518668
precision	0.791353	0.518668
recall	0.863059	0.518668
mcc	0.592415	0.518668

Confusion matrix (at threshold=0.518668)

	Predicted as long	Predicted as short
Labeled as long	1428	555
Labeled as short	334	2105

Learning curves

Decision Tree

Tree #1

Rules

if (Total Costs <= 7840.645) and (CCS Diagnosis Code <= 454.5) and (Total Costs <= 4864.71) then class: short (proba: 95.04%) | based on 3,269 samples

if (Total Costs <= 7840.645) and (CCS Diagnosis Code <= 454.5) and (Total Costs > 4864.71) then class: short (proba: 74.99%) | based on 2,363 samples

if (Total Costs > 7840.645) and (APR Severity of Illness Code > 2.5) and (Total Costs > 11670.13) then class: long (proba: 92.99%) | based on 2,211 samples

if (Total Costs > 7840.645) and (APR Severity of Illness Code <= 2.5) and (APR Medical Surgical Description <= 1.5) then class: long (proba: 71.26%) | based on 2,150 samples

if (Total Costs > 7840.645) and (APR Severity of Illness Code <= 2.5) and (APR Medical Surgical Description > 1.5) then class: short (proba: 57.91%) | based on 2,036 samples

if (Total Costs > 7840.645) and (APR Severity of Illness Code > 2.5) and (Total Costs <= 11670.13) then class: long (proba: 69.95%) | based on 599 samples

if (Total Costs <= 7840.645) and (CCS Diagnosis Code > 454.5) and (Total Costs > 3798.055) then class: long (proba: 76.8%) | based on 362 samples

if (Total Costs <= 7840.645) and (CCS Diagnosis Code > 454.5) and (Total Costs <= 3798.055) then class: short (proba: 80.73%) | based on 275 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

0.7 seconds

Metric details

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Confusion matrix (at threshold=0.496442)

	Predicted as long	Predicted as short
Labeled as long	0	1983
Labeled as short	0	2439

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

23.3 seconds

Metric details

	score	threshold
logloss	0.380921	nan
auc	0.913052	nan
f1	0.860528	0.415301
accuracy	0.83763	0.424104
precision	0.996815	0.962424
recall	1	0.0459357
mcc	0.672817	0.415301

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.380921	nan
auc	0.913052	nan
f1	0.859711	0.424104
accuracy	0.83763	0.424104
precision	0.821202	0.424104
recall	0.902009	0.424104
mcc	0.672165	0.424104

Confusion matrix (at threshold=0.424104)

	Predicted as long	Predicted as short
Labeled as long	1504	479
Labeled as short	239	2200

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	5

Metric details

	score	threshold
logloss	0.259062	nan
auc	0.958834	nan
f1	0.904165	0.452821
accuracy	0.893487	0.556852
precision	0.996491	0.994703
recall	1	0.000147622
mcc	0.785646	0.556852

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.259062	nan
auc	0.958834	nan
f1	0.902181	0.556852
accuracy	0.893487	0.556852
precision	0.914141	0.556852
recall	0.890529	0.556852
mcc	0.785646	0.556852

Confusion matrix (at threshold=0.556852)

	Predicted as long	Predicted as short
Labeled as long	1779	204
Labeled as short	267	2172

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

8.8 seconds

Metric details

	score	threshold
logloss	0.362407	nan
auc	0.92855	nan
f1	0.870787	0.470751
accuracy	0.854817	0.49367
precision	0.982955	0.993535
recall	1	7.45466e-08
mcc	0.705898	0.500039

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.362407	nan
auc	0.92855	nan
f1	0.870512	0.49367
accuracy	0.854817	0.49367
precision	0.856689	0.49367
recall	0.884789	0.49367
mcc	0.705878	0.49367

Confusion matrix (at threshold=0.49367)

	Predicted as long	Predicted as short
Labeled as long	1622	361
Labeled as short	281	2158